[REVIEW] Don't identify decimals as strings. #7710
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vyasr
added
3 - Ready for Review
Codecov Report
@@ Coverage Diff @@
## branch-0.19 #7710 +/- ##
===============================================
+ Coverage 81.86% 82.51% +0.64%
===============================================
Files 101 101
Lines 16884 17450 +566
===============================================
+ Hits 13822 14398 +576
+ Misses 3062 3052 -10
Continue to review full report at Codecov.
|
kkraus14
reviewed
Mar 24, 2021
vyasr
added
2 - In Progress
|
Looks like this is a slightly more subtle problem. There's more to be done here to actually enable decimal operations; the way that they're currently be rejected is problematic (what I fixed above), but they are currently being rejected intentionally and I'll have to do a bit more work to actually enable them. |
vyasr
changed the title
vyasr
changed the title
vyasr
added
3 - Ready for Review
kkraus14
approved these changes
Mar 25, 2021
kkraus14
added
5 - Ready to Merge
|
@gpucibot merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As documented in this pandas issue,
is_string_typefor pandas is not strict and will characterize a whole bunch of things as strings that aren't. For our purposes, this is problematic because basically all subclasses ofExtensionDTypewill be classified as strings by that function. This is definitely not appropriate, so I modified our version ofis_string_dtypeto explicitly reject all of our extension dtypes (previously it was only excluding categorical types). I'm not 100% confident that no other parts of the code base rely on the current (erroneous) behavior, but the cudf tests all passed for me locally and my attempt to trace all calls ofutils.is_string_dtypeall look to be places where the change gives more correct behavior, so I think our best bet is to just move forward with this change. Any problems that result from this change in the future due to other code relying on the current behavior should probably be characterized as bugs in the calling code and fixed there. The same goes for for external codes that relied on this behavior; this change is potentially breaking for them as well, but again is something that they should be addressing.